Detection and localization of specular surfaces using image motion cues

Cataloged from PDF version of article.Successful identification of specularities in an image can be crucial for an artificial vision system when extracting the semantic content of an image or while interacting with the environment. We developed an algorithm that relies on scale and rotation invariant feature extraction techniques and uses motion cues to detect and localize specular surfaces. Appearance change in feature vectors is used to quantify the appearance distortion on specular surfaces, which has previously been shown to be a powerful indicator for specularity (Doerschner et al. in Curr Biol, 2011). The algorithm combines epipolar deviations (Swaminathan et al. in Lect Notes Comput Sci 2350:508-523, 2002) and appearance distortion, and succeeds in localizing specular objects in computer-rendered and real scenes, across a wide range of camera motions and speeds, object sizes and shapes, and performs well under image noise and blur conditions. © 2014 Springer-Verlag Berlin Heidelberg

Effects of surface reflectance and 3D shape on perceived rotation axis

Cataloged from PDF version of article.Surface specularity distorts the optic flow generated by a moving object in a way that provides important cues for identifying surface material properties (Doerschner, Fleming et al., 2011). Here we show that specular flow can also affect the perceived rotation axis of objects. In three experiments, we investigate how threedimensional shape and surface material interact to affect the perceived rotation axis of unfamiliar irregularly shaped and isotropic objects. We analyze observers' patterns of errors in a rotation axis estimation task under four surface material conditions: shiny, matte textured, matte untextured, and silhouette. In addition to the expected large perceptual errors in the silhouette condition, we find that the patterns of errors for the other three material conditions differ from each other and across shape category, yielding the largest differences in error magnitude between shiny and matte, textured isotropic objects. Rotation axis estimation is a crucial implicit computational step to perceive structure from motion; therefore, we test whether a structure from a motion-based model can predict the perceived rotation axis for shiny and matte, textured objects. Our model's predictions closely follow observers' data, even yielding the same reflectance-specific perceptual errors. Unlike previous work (Caudek & Domini, 1998), our model does not rely on the assumption of affine image transformations; however, a limitation of our approach is its reliance on projected correspondence, thus having difficulty in accounting for the perceived rotation axis of smooth shaded objects and silhouettes. In general, our findings are in line with earlier research that demonstrated that shape from motion can be extracted based on several different types of optical deformation (Koenderink & Van Doorn, 1976; Norman & Todd, 1994; Norman, Todd, & Orban, 2004; Pollick, Nishida, Koike, & Kawato, 1994; Todd, 1985). © 2013 Arvo

Object rigidity and reflectivity identification based on motion analysis

Rigidity and reflectivity are important properties of objects, identifying these properties is a fundamental problem for many computer vision applications like motion and tracking. In this paper, we extend our previous work to propose a motion analysis based approach for detecting the object's rigidity and reflectivity. This approach consists of two steps. The first step aims to identify object rigidity based on motion estimation and optic flow matching. The second step is to classify specular rigid and diffuse rigid objects using structure from motion and Procrustes analysis. We show how rigid bodies can be detected without knowing any prior motion information by using a mutual information based matching method. In addition, we use a statistic way to set thresholds for rigidity classification. Presented results demonstrate that our approach can efficiently classify the rigidity and reflectivity of an object. © 2010 IEEE

Estimating the glossiness transfer function induced by illumination change and testing its transitivity

The light reflected from a glossy surface depends on the reflectance properties of that surface as well as the flow of light in the scene, the light field. We asked four observers to compare the glossiness of pairs of surfaces under two different realword light fields, and used this data to estimate a transfer function that captures how perceived glossiness is remapped in changing from one real-world light field to a second. We wished to determine the form of the transfer function and to test whether for any set of three light fields the transfer function from light field 1 to light field 2 and the transfer function from light field 2 to light field 3 could be used to predict the glossiness transfer function from light field 1 to light field 3. Observers' estimated glossiness transfer functions for three sets of light fields were best described by a linear model. The estimated transfer functions exhibited the expected transitivity pattern for three out of four observers. The failure of transitivity for one observer, while significant, was less than 12.5% of the gloss range. © ARVO

Differences in illumination estimation in #thedress

We investigated whether people who report different colors for #thedress do so because they have different assumptions about the illumination in #thedress scene. We introduced a spherical illumination probe (Koenderink, Pont, van Doorn, Kappers, & Todd, 2007) into the original photograph, placed in fore-, or background of the scene and-for each location-let observers manipulate the probe's chromaticity, intensity and the direction of the illumination. Their task was to adjust the probe such that it would appear as a white sphere in the scene. When the probe was located in the foreground, observers who reported the dress to be white (white perceivers) tended to produce bluer adjustments than observers who reported it as blue (blue perceivers). Blue perceivers tended to perceive the illumination as less chromatic. There were no differences in chromaticity settings between perceiver types for the probe placed in the background. Perceiver types also did not differ in their illumination intensity and direction estimates across probe locations. These results provide direct support for the idea that the ambiguity in the perceived color of the dress can be explained by the different assumptions that people have about the illumination chromaticity in the foreground of the scene. In a second experiment we explore the possibility that blue perceivers might overall be less sensitive to contextual cues, and measure white and blue perceivers' dress color matches and labels for manipulated versions of the original photo. Results indeed confirm that contextual cues predominantly affect white perceivers

Human visual cortical responses to specular and matte motion flows

Determining the compositional properties of surfaces in the environment is an important visual capacity. One such property is specular reflectance, which encompasses the range from matte to shiny surfaces. Visual estimation of specular reflectance can be informed by characteristic motion profiles; a surface with a specular reflectance that is difficult to determine while static can be confidently disambiguated when set in motion. Here, we used fMRI to trace the sensitivity of human visual cortex to such motion cues, both with and without photometric cues to specular reflectance. Participants viewed rotating blob-like objects that were rendered as images (photometric) or dots (kinematic) with either matte-consistent or shiny-consistent specular reflectance profiles. We were unable to identify any areas in low and mid-level human visual cortex that responded preferentially to surface specular reflectance from motion. However, univariate and multivariate analyses identified several visual areas; V1, V2, V3, V3A/B, and hMT+, capable of differentiating shiny from matte surface flows. These results indicate that the machinery for extracting kinematic cues is present in human visual cortex, but the areas involved in integrating such information with the photometric cues necessary for surface specular reflectance remain unclear. © 2015 Kam, Mannion, Lee, Doerschner and Kersten

Perceived glossiness in high dynamic range scenes

We investigated how spatial pattern, background, and dynamic range affect perceived gloss in brightly lit real scenes. Observers viewed spherical objects against uniform backgrounds. There were three possible objects. Two were black matte spheres with circular matte white dots painted on them (matte-dot spheres). The third sphere was painted glossy black (glossy black sphere). Backgrounds were either black or white matte, and observers saw each of the objects in turn on each background. Scenes were illuminated by an intense collimated source. On each trial, observers matched the apparent albedo of the sphere to an albedo reference scale and its apparent gloss to a gloss reference scale. We found that mattedot spheres and the black glossy sphere were perceived as glossy on both backgrounds. All spheres were judged to be significantly glossier when in front of the black background. In contrast with previous research using conventional computer displays, we find that background markedly affects perceived gloss. This finding is surprising because darker surfaces are normally perceived as glossier (F. Pellacini, J. A. Ferwerda, & D. P. Greenberg, 2000). We conjecture that there are cues to surface material signaling glossiness present in high dynamic range scenes that are absent or weak in scenes presented using conventional computer displays. © ARVO

Effects of surface reflectance and 3D shape on perceived rotation axis

Surface specularity distorts the optic flow generated by a moving object in a way that provides important cues for identifying surface material properties (Doerschner, Fleming et al., 2011). Here we show that specular flow can also affect the perceived rotation axis of objects. In three experiments, we investigate how threedimensional shape and surface material interact to affect the perceived rotation axis of unfamiliar irregularly shaped and isotropic objects. We analyze observers' patterns of errors in a rotation axis estimation task under four surface material conditions: shiny, matte textured, matte untextured, and silhouette. In addition to the expected large perceptual errors in the silhouette condition, we find that the patterns of errors for the other three material conditions differ from each other and across shape category, yielding the largest differences in error magnitude between shiny and matte, textured isotropic objects. Rotation axis estimation is a crucial implicit computational step to perceive structure from motion; therefore, we test whether a structure from a motion-based model can predict the perceived rotation axis for shiny and matte, textured objects. Our model's predictions closely follow observers' data, even yielding the same reflectance-specific perceptual errors. Unlike previous work (Caudek & Domini, 1998), our model does not rely on the assumption of affine image transformations; however, a limitation of our approach is its reliance on projected correspondence, thus having difficulty in accounting for the perceived rotation axis of smooth shaded objects and silhouettes. In general, our findings are in line with earlier research that demonstrated that shape from motion can be extracted based on several different types of optical deformation (Koenderink & Van Doorn, 1976; Norman & Todd, 1994; Norman, Todd, & Orban, 2004; Pollick, Nishida, Koike, & Kawato, 1994; Todd, 1985). © 2013 Arvo

Surface color perception and light field estimation in 3D scenes

The light field The spectral power distribution of the light emitted by the Sun is almost constant. The variations in daylight (Figure 13.1) that we experience over the course of a day and with changes in seasons are due to the interaction of sunlight with the Earth’s atmosphere (Henderson, 1977). The resulting spectral distribution of daylight across the sky is typically spatially inhomogeneous and constantly changing (Lee and Hernández-Andr’s, 2005a, b). The light arriving at each small patch of surface in the scene depends in general on the patch’s location and orientation in the scene. Furthermore, objects in the scene create shadows or act as secondary light sources, adding further complexity to the light field (Gershun, 1936/1939; see also Adelson and Bergen, 1991) that describes the spectral power distribution of the light arriving from every direction at every point in the scene. The light field captures what a radiospectrophotometer placed at each point in the scene, pointing in all possible directions, would record (Figure 13.2). When the light field is inhomogeneous, the light absorbed and reradiated by a matte smooth surface patch can vary markedly with the orientation or location of the patch in the scene. In Figure 13.3, for example, we illustrate the wide range of the light emitted by identical rectangular achromatic matte surfaces at many orientations, illuminated by a distant neutral, collimated light source. © Cambridge University Press 2011

Effects of surface reflectance on local second order shape estimation in dynamic scenes

In dynamic scenes, relative motion between the object, the observer, and/or the environment projects as dynamic visual information onto the retina (optic flow) that facilitates 3D shape perception. When the object is diffusely reflective, e.g. a matte painted surface, this optic flow is directly linked to object shape, a property found at the foundations of most traditional shape-from-motion (SfM) schemes. When the object is specular, the corresponding specular flow is related to shape curvature, a regime change that challenges the visual system to determine concurrently both the shape and the distortions of the (sometimes unknown) environment reflected from its surface. While human observers are able to judge the global 3D shape of most specular objects, shape-from-specular-flow (SFSF) is not veridical. In fact, recent studies have also shown systematic biases in the perceived motion of such objects. Here we focus on the perception of local shape from specular flow and compare it to that of matte-textured rotating objects. Observers judged local surface shape by adjusting a rotation and scale invariant shape index probe. Compared to shape judgments of static objects we find that object motion decreases intra-observer variability in local shape estimation. Moreover, object motion introduces systematic changes in perceived shape between matte-textured and specular conditions. Taken together, this study provides a new insight toward the contribution of motion and surface material to local shape perception. © 2015 The Authors